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Abstract 

We study sparse approximation by greedy algorithms. We prove 
the Lebesgue-type inequalities for the Weak Chebyshev Greedy Algo- 
rithm (WCGA), a generalization of the Weak Orthogonal Matching 
Pursuit to the case of a Banach space. The main novelty of these re- 
sults is a Banach space setting instead of a Hilbert space setting. The 
results are proved for redundant dictionaries satisfying certain condi- 
tions. Then we apply these general results to the case of bases. In 
particular, we prove that the WCGA provides almost optimal sparse 
approximation for the trigonometric system in L p , 2 < p < oo. 

1 Introduction 

This paper is devoted to theoretical aspects of sparse approximation. The 
main motivation for the study of sparse approximation is that many real 
world signals can be well approximated by sparse ones. Sparse approximation 
automatically implies a need for nonlinear approximation, in particular, for 
greedy approximation. We give a brief description of a sparse approximation 
problem. In a general setting we are working in a Banach space X with a 
redundant system of elements T> (dictionary D). There is a solid justification 
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of importance of a Banach space setting in numerical analysis in general and 
in sparse approximation in particular (see, for instance, [TT], Preface, and 
[?]). An element (function, signal) / e A is said to be A-sparse with respect 
to V if it has a representation / = J2iLi x i9i-, g% €. T>, i = 1, . . . , K. The set 
of all A-sparse elements is denoted by For a given element f we 

introduce the error of best m-term approximation 

a m (fo,V) := inf ||/o-/||. 

We are interested in the following fundamental problem of sparse approxi- 
mation. 

Problem. How to design a practical algorithm that builds sparse ap- 
proximations comparable to best m-term approximations? 

We demonstrate in this paper that the Weak Chebyshev Greedy Algo- 
rithm (WCGA) which we define momentarily is a solution to the above prob- 
lem. This paper is devoted to the Banach space setting. Let A be a real 
Banach space with norm || ■ || := || ■ We say that a set of elements (func- 
tions) D from A is a dictionary if each g e T> has norm one (||g|| = 1), and 
the closure of spanD is A. For a nonzero element g G A we let F g denote a 
norming (peak) functional for g: 

\\F g \\x* = 1, F g (g) = \\g\\ x - 

The existence of such a functional is guaranteed by the Hahn-Banach theo- 
rem. 

Let r := {tfc}^ =1 be a given weakness sequence of nonnegative num- 
bers tk < 1, k — 1, We define the Weak Chebyshev Greedy Algorithm 

(WCGA) (see 0) as a generalization for Banach spaces of the Weak Orthogo- 
nal Matching Pursuit (WOMP). In a Hilbert space the WCGA coincides with 
the WOMP. The WOPM is very popular in signal processing, in particular, 
in compressed sensing. We study in detail the WCGA in this paper. 

Weak Chebyshev Greedy Algorithm (WCGA). Let f be given. 
Then for each m > 1 we have the following inductive definition. 

(1) tp m := y?^ G D is any element satisfying 

\ F fm-i(fm)\ ttnSwplFf^ig)]. 

geT) 

(2) Define 

$ m := $^ := span{<^}™ =1 , 
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and define G m := to be the best approximant to f from $ m . 
(3) Let 

m ■ Jrn • JO ( ~ T m- 

In this paper we only consider the case when % = t G (0, 1], k = 1, 2, 

The trigonometric system is a classical system that is known to be difficult 
to study. In this paper we study among other problems the problem of 
nonlinear sparse approximation with respect to it. Let 1ZT denote the real 
trigonometric system 1, sin 2nx, cos 2nx, . . . on [0,1] and let TZT P to be its 
version normalized in L p ([0, 1]). Denote VST p := 7ZT P x ■ ■ • x 70~ p the d- 
variate trigonometric system. We need to consider the real trigonometric 
system because the algorithm WCGA is well studied for the real Banach 
space. In order to illustrate performance of the WCGA we discuss in this 
section the above mentioned problem for the trigonometric system. There is 
a natural algorithm, the Thresholding Greedy Algorithm (TGA), that can 
be considered for the above problem. We give a definition of the TGA for a 
general basis Let a Banach space X, with a normalized basis \1/ = {ipk}kLi, 
be given. We consider the following greedy algorithm. For a given element 
/ G X we consider the expansion 

oo 

fe=l 

For an element / G X we say that a permutation p of the positive integers 
is decreasing if 

M/)|> W/)|>..., (i.2) 

where p(j) = kj, j = 1,2, ... , and write p G D(f). If the inequalities are 
strict in ( II. 2p . then D(f) consists of only one permutation. We define the 
mth greedy approximant of /, with regard to the basis ^ corresponding to 
a permutation p G D(f), by the formula 

m 

G m (f,V) :=G m (f,^,p):=^2c kj {f)i, kj . 

3=1 

The following Lebesgue-type inequality was proved in [8]. 
Theorem 1.1. For each f G L p ([0, l] d ) we have 

||/ - G m (f,KTl)\\ p ^ C(d)m h ^o- m (f,KT d p ) p , 1 < V < oo, 
where h(p) := 1 1/2 — l/p\. 
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It was also proved in [5] that the above inequality is sharp. 

Remark 1.1. There is a positive absolute constant C such that for each m 
and 1 < p < oo there exists a function f ^ with the property 

\\G m (f,KT p )\\ p >Cm h <*)\\f\\ p . (1.3) 

Remark 1 1 . 1 1 shows that the TGA does not work well for the trigonometric 
system in L p , p ^ 2. This leads to a natural attempt to consider some other 
algorithms that may have some advantages over the TGA in the case of the 
trigonometric system. In this paper we discuss the performance of the Weak 
Chebyshev Greedy Algorithm (WCGA) with respect to the trigonometric 
system. We prove here the following Lebesgue-type inequality for the WCGA 
(see Example 2 in Section 4). 

Theorem 1.2. Let T> be the normalized in L p , 2 < p < oo, real d-variate 
trigonometric system. Then for any f G L p the WCGA with weakness 
parameter t gives 

\\fc(t , P ,d)mln(m+l) \\p 

<Ca m (f ,V) p . (1.4) 

The Open Problem 7.1 (p. 91) from JTHj asks if (11.41) holds without an 
extra ln(m + 1) factor. Theorem 11.21 is the first result on the Lebesgue-type 
inequalities for the WCGA with respect to the trigonometric system. It 
provides a progress in solving the above mentioned open problem, but the 
problem is still open. 

Theorem 11.21 shows that the WCGA is very well designed for the trigono- 
metric system. We show in Example 1 of Section 4 that an analog of (11.41) 
holds for uniformly bounded orthogonal systems. We note that it is known 
(see [11]) that the TGA is very well designed for bases L p -equivalent to the 
Haar basis, 1 < p < oo. We discuss performance of the WCGA in more 
detail in Section 5. 

The proof of Theorem 1 1 . 21 uses technique developed for proving the Lebesgue- 
type inequalities for redundant dictionaries with special properties. We 
present these results in Sections 2 and 3. These results are an extension 
of earlier results from [5]. In Section 4 we test the power of general results 
from Section 2 on specific dictionaries, namely, on bases. Section 4 provides a 
number of examples, including the trigonometric system, were the technique 
from Sections 2 and 3 can be successfully applied. In particular, results from 
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Section 4 demonstrate that the general technique from Sections 2 and 3 pro- 
vides almost optimal m-term approximation results for uniformly bounded 
orthogonal systems (see Example 1). Example 7 shows that an extra as- 
sumption that a uniformly bounded orthogonal system ^ is a quasi-greedy 
basis allows us to improve inequality ( \1A\i : 

\\fc(t, P ,^)m In ln(m+3) 1 1 p 

2 Lebesgue-type inequalities. General results. 

A very important advantage of the WCGA is its convergence and rate of 
convergence properties. The WCGA is well defined for all m. Moreover, it 
is known (see [H] and [UJ) that the WCGA with r = {t} converges for all f 
in all uniformly smooth Banach spaces with respect to any dictionary. That 
is, when X is a real Banach space and the modulus of smoothness of X is 
defined as follows 

p(u) := - sup | ||x + uy\\ + \\x — uy\\ — 2| , (2.1) 

2 x,y; ||a[|=||j/||=l 

then the uniformly smooth Banach space is the one with p(u)/u — > when 
it — y 0. 

We discuss here the Lebesgue-type inequalities for the WCGA with r = 
{t}, t G (0,1]. For notational convenience we consider here a countable 
dictionary T> = {gi}^. The following assumptions Al and A2 were used in 
[5]. For a given / let sparse element (signal) 

f ■= f e = ^Xi9i 

be such that ||/ — / e || < e an d \T\ = K. For A cT denote 

!a ■= fl ■= ^Xigi- 

Al. We say that / = Y2ieT x i9i satisfies the Nikol'skii-type t\X inequal- 
ity with parameter r if 

£>i| < dl^m/All, ACT, r>l/2. (2.2) 

i€A 
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We say that a dictionary T> has the Nikol'skii-type £\X property with param- 
eters K, r if any i^-sparse element satisfies the Nikol'skii-type £\X inequality 
with parameter r. 

A2. We say that / = YlieT x i9i has incoherence property with parame- 
ters D and U if for any AcT and any A such that A n A = 0, | A\ + |A| < D 
we have for any {cj} 

\\f A -J2cigi\\ ^U-'WhW. (2.3) 

We say that a dictionary T> is (K, Z})-unconditional with a constant U if for 
any / = J2ieT x i9* with 1^1 — K inequality (I2.3P holds. 

The term unconditional in A2 is justified by the following remark. The 
above definition of (K, Z})-unconditional dictionary is equivalent to the fol- 
lowing definition. Let T> be such that any subsystem of D distinct elements 
ei, . . . , en from T> is linearly independent and for any Ac[l,D] with | A\ < K 
and any coefficients {q} we have 

D 

ii y^c^n < u\\ y^Qeiii. 

ieA i=l 

It is convenient for us to use the following assumption A3 which is a 
corollary of assumptions Al and A2. 

A3. We say that / = Ylii&T x i9i has t\ incoherence property with pa- 
rameters D, V, and r if for any A C T and any A such that A fl A = 0, 
|^4| + |A| < D we have for any {q} 

X>i|< Wll/A-I>fcll- (2-4) 

ieA ieA 

A dictionary P has i\ incoherence property with parameters K, D, V, and 
r if for any A C B, \A\ < K, \B\ < D we have for any {ci}i e B 

ieA ieB 

It is clear that Al and A2 imply A3 with V = C%U. Also, A3 implies Al 
with C\ = V and A2 with U = VK T . Obviously, we can restrict ourselves 
to r < 1. 

We now proceed to main results of this paper on the WCGA with respect 
to redundant dictionaries. The following Theorem 12.11 in the case q = 2 was 
proved in [3]. 
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Theorem 2.1. Let X be a Banach space with p{u) < 7M 9 , 1 < q < 2. 
Suppose K -sparse f e satisfies Al, A2 and ||/ — f e \\ — £■ Then the WCGA 
with weakness parameter t applied to fo provides 

ll/c (t , 7 A)^'Mt/+i)Wll<Ce for K + Cit^CjUi'HU + l)^' <D 
with an absolute constant C . 

It was pointed out in [3] that Theorem 12 . 1 1 provides a corollary for Hilbert 
spaces that gives sufficient conditions somewhat weaker than the known RIP 
conditions on V for the Lebesgue-type inequality to hold. We formulate the 
corresponding definitions and results. Let T> be the Riesz dictionary with 
depth D and parameter 5 G (0, 1). This class of dictionaries is a generaliza- 
tion of the class of classical Riesz bases. We give a definition in a general 
Hilbert space (see [TT], p. 306). 

Definition 2.1. A dictionary T> is called the Riesz dictionary with depth D 
and parameter 5 G (0, 1) if, for any D distinct elements ei, . . . , of the 
dictionary and any coefficients a = (ax, . . . , an), we have 

D 

(1 - S)\\a\\ 2 2 < ||$> ej || 2 < (l + «5)||a||jj. (2.5) 

i=l 

We denote the class of Riesz dictionaries with depth D and parameter 5 G 
(0,1) byR(D,5). 

The term Riesz dictionary with depth D and parameter 5 G (0, 1) is 
another name for a dictionary satisfying the Restricted Isometry Property 
(RIP) with parameters D and 5. The following simple lemma holds. 

Lemma 2.1. Let V G R(D,5) and let ej G V, j = 1, ...,s. For f = 

J2t=i a i e i an d A c {1j • • • > s } denote 

Ifs<D then 

ll^(/)|| 2 <(l + 5)(l-5)- 1 ||/|| 2 . 

Lemma 12.11 implies that if T> G R(D, 5) then it is (D, /^-unconditional 
with a constant U = (1 + 5) l l 2 {\ - 5)- 1 ' 2 . 
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Theorem 2.2. Let X be a Hilbert space. Suppose K-sparse f £ satisfies A2 
and || Jo — — e - Then the WOMP with weakness parameter t applied to 
fo provides 

\\fc(t,u)K\\<Ce for K + C(t,U)K<D 
with an absolute constant C . 

Theorem 12.21 implies the following corollaries. 

Corollary 2.1. Let X be a Hilbert space. Suppose any K-sparse f satisfies 
A 2. Then the WOMP with weakness parameter t applied to fo provides 

\\fc(t,u)K\\<Ca K (f ,V) for K + C(t,U)K<D 

with an absolute constant C . 

Corollary 2.2. Let X be a Hilbert space. Suppose T> G R(D,5). Then the 
WOMP with weakness parameter t applied to fo provides 

\\fc(t,s)K\\<Ca K (f ,V) for K + C{t,5)K<D 

with an absolute constant C . 

We emphasized in [5] that in Theorem 12. II we impose our conditions on an 
individual function f e . It may happen that the dictionary does not have the 
Nikol'skii i\X property and (K, D)-unconditionality but the given f can be 
approximated by f e which does satisfy assumptions Al and A2. Even in the 
case of a Hilbert space the above results from [5] add something new to the 
study based on the RIP property of a dictionary. First of all, Theorem 12.21 
shows that it is sufficient to impose assumption A2 on f € in order to obtain 
exact recovery and the Lebesgue-type inequality results. Second, Corollary 
12.11 shows that the condition A2, which is weaker than the RIP condition, is 
sufficient for exact recovery and the Lebesgue-type inequality results. Third, 
Corollary 12.21 shows that even if we impose our assumptions in terms of RIP 
we do not need to assume that 5 < 5 . In fact, the result works for all 5 < 1 
with parameters depending on 5. 

Theorem 12.11 follows from the combination of Theorems 12.31 and 12.41 In 
case q = 2 these theorems were proved in [5]. 
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Theorem 2.3. Let X be a Banach space with p(u) < '"fu q , 1 < q < 2. 
Suppose for a given f we have \\fo — f e \\ < e with K -sparse f := f e satisfying 
A3. Then for any k > we have for K + m < D 

||/m|| < \\fk\\ exp — — +2e, g — 



i^rg' 7 ' * q _ 1 



where c\ := — 

2(167) 



In all theorems that follow we assume rq' > 1. 

Theorem 2.4. Le£ I fe o Banach space with p{u) < ^vfi , 1 < g < 2. 
Suppose K-sparse f e satisfies Al, A2 and ||/ — < e. T/ien i/ie WCGA 
with weakness parameter t applied to /o provides 

\\fam'Hu + i) K rA<CUe for K + C'U q ' la(U + l)K rq ' < D 

with an absolute constant C and C = C 2 (g)79 Tri Cf t~ q ' . 

We formulate an immediate corollary of Theorem 12.41 with e = 0. 

Corollary 2.3. Let X be a Banach space with p(u) < 7M 9 . Suppose K- 
sparse f satisfies Al, A 2. Then the WCGA with weakness parameter t 
applied to f recovers it exactly after C'U q \n(U + l)K rq iterations under 
condition K + C'U q ' \n{U + l)K rq ' < D. 

We formulate versions of Theorem 12.41 with assumptions Al, A 2 replaced 
by a single assumption A3 and replaced by two assumptions A2 and A3. 
The corresponding modifications in the proofs go as in the proof of Theorem 



Theorem 2.5. Let X be a Banach space with p(u) < 7M 9 , 1 < q < 2. 
Suppose K-sparse / e satisfies A3 and \\fo — < e. Then the WCGA with 
weakness parameter t applied to f provides 

\\fc(t, 7 , q )v«'HVK)Kr4<CVK r e for K + C(t n ,q)V q 'HVK)K rq ' <D 
with an absolute constant C and C(t, 7, q) = C2(g)7^ rT t~ 9 '. 
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Theorem 2.6. Let X be a Banach space with p(u) < 7W 9 , 1 < q < 2. 
Suppose K -sparse f e satisfies A2, A3 and \\f — f e \\ < e. T/ien £/ie WCGA 
with weakness parameter t applied to fo provides 

\\fc(t, 7 , q) v«> H u + i)KW II < CUe for K + C(t, 7, q)V q ' \n(U + l)K r «' < D 

with an absolute constant C and C(t, 7, q) = C2(q)'y~t~ q ' . 

Theorems 12.51 and 12.31 imply the following analog of Theorem 12.11 

Theorem 2.7. Let X be a Banach space with p{u) < ^u q , 1 < q < 2. 

Suppose K -sparse f e satisfies A3 and \\f — f e \\ < e. Then the WCGA with 
weakness parameter t applied to fo provides 

Wfcit^v*' H vk)k^ II < Ce for K + C(t, 7, q)V>' \n(VK)K rq ' < D 

1 , 

with an absolute constant C and C(t,j, q) = C2{q)^i q ^ 1 t^ q . 

The following edition of Theorems 12.11 and 12.71 is also useful in applica- 
tions. It follows from Theorems 12.61 and 12.31 

Theorem 2.8. Let X be a Banach space with p(u) < 7U 9 , 1 < q < 2. 
Suppose K-sparse f e satisfies A2, A3 and \\fo — f e \\ < e. Then the WCGA 
with weakness parameter t applied to fo provides 

\\fc { t n , q) v«>HU + i)K^\\<Ce for K + C(t, 7, q)V q ' ln([7 + l)K rq ' < D 
with an absolute constant C and C(t,7,g) = C2(q)'J q ~ I t^ q ' . 



3 Proofs 

We begin with a proof of Theorem [2? 
Proof. Let 

/: /' »,</,. \T\ = K, 9i eV. 

Denote by T m the set of indices of gj G D picked by the WCGA after m 
iterations, T m :— T\ T m . Denote by Ai(TJ) the closure in X of the convex 
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hull of the symmetrized dictionary V ± := {±g,g G D}. We will bound ||/ m || 
from above. Assume j| /"^—i || > e. Let m > k. We bound from below 

S m := sup 1^/^(0) | . 
(f>eAi(T>) 

Denote A m := r m_1 . Then 

S m > ^/ m -i(/A m /||/A m ||l), 

where := J2ieA Next, by Lemma 6.9, p. 342, from [11] we obtain 

F fm AfA m ) = F fm _ 1 {r)> ||/ m _i||-e. 

Thus 

fi^^ll/^llr'dl/m-ill-e)- (3-1) 
From the definition of the modulus of smoothness we have for any A 

ll/m-i-A^II + ll/^-i+A^II <2\\f m ^\\ (l + p (-A—\j (3.2) 

and by (1) from the definition of the WCGA and Lemma 6.10 from [IT] , p. 
343, we get 

\Ff m -i(fm)\ > tswplFf^ig)] = 
t sup \ F f m -M\=tS m . 

4>EAi(V) 

Then either (w) > tS m or i 7 / m _ 1 (— <f m ) > £<S»n- Both cases are treated 
in the same way. We demonstrate the case i 7 / m _ 1 (^m) > We have for 
A > 

+ Ay? m || > Ff m _ x (f m -.\ + Ay? m ) > ||/ m _i|| + At^m. 
From here and from (13.21) we obtain 
\\f m \\ < ||/«-i-A^ m || < ||/ m _i||+ inf(-At-5 m + 2||/ w _ 1 ||p(A/||/ m _ 1 ||)). 

We discuss here the case p{u) < -yu q . Using (13.11) we get 

/m < ||/m-l|| I" n 7 ii +27 



||,/,4 m ||l ||/ro-l||V ||/A m ||l 
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Let Ai be a solution of 



2||/.4,„lli '\\f m -i\\«' V-*tII/a»ih. 
Our assumption (I2.4p gives 

< WT(||/ - G m _!|| + ||/o - f ||) < ^(11/^11 + e) 

Specify 

\l&y{VK r )*J 

Then, using ||/ m _i|| > e we get 

Ay- 1 ii/aJI? 



V 4||/ m _ 1 ||«(W)« 



< 1 



and obtain 



II/,,, II < II/,,- 1 II | 1 r J + ? • 

2{lQ 1 ) — {VK r Y) (l6-f) — (VK r y 

Denote C\ := . Then 

2(167) 9 =T v« / 



ll/JI<Mex P (-^ 7 ^')+2 e - 



□ 



We proceed to a proof of Theorem 12.41 Modifications of this proof which 
are in a style of the above proof of Theorem 12.31 give Theorems 12.51 and 12.61 



Proof. We use the above notations T m and T m :— T \ T m . Let k > be 
fixed. Suppose 

2 »-i < |pfc| < 2 ". 

For j = 1, 2, . . . , n,n + 1 consider the following pairs of sets Aj, Bf A n+ \ = 
T k , B n+1 = 0; for j < n, Aj := T k \ Bj with Bj C T k is such that \BA > 
\T k \ - 2J" 1 and for any set J C T k with \J\ > \T k \ - <2?~ l we have 

\\f Bj \\ < Wfjl 
12 



Then 



We note that this implies that if for some Q C T k we have 

||/ Q ||<||/ B ,|| then |Q| < |r fe | - V-\ (3.3) 

For a given b > 1, to be specified later, denote by L the index such that 
(Bo := T k ) 

WfBj<b\\f Bl \\, 

ll/*ll<&ll/*ll, 

||/b £ _J<&||/b £ _J, 
WfB L J\>b\\f BL \\. 

II/^II^^-^II/b^xII, j = l,2,...,L. (3.4) 

We now proceed to a general step. Let m > k and let A, B C T k be such 
that A = T k \B. As above we bound S m from below. It is clear that S m > 0. 
Denote A m := A n r m_1 . Then 

S m > Ff m _ x (f Am /\\f Am \\l). 

Next, 

Ff m -x{fA m ) = Ff m -i(fA m + Ib - /b)- 

Then + f B = f e - f\ with F/^^/a) = 0. Moreover, it is easy to see 
that Ff^^p) > \\fm-i\\ - e. Therefore, 

F fm AfA m + /b - f B ) > ||/™-i|| - c - ||/b||. 

Thus 

5 m > H/Ajlr 1 max(0, ll/^H c || f B ||). 

By (T22D we get 
Then 

Q v. Il/m-i|| ~ H/bII - e /„ -s 

m - cMmz\\ ■ ( } 
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From the definition of the modulus of smoothness we have for any A 
||/ m _i - \<p m \\ + ||/ m _i + A<^ m || < 2||/ m _ 1 ||(l + p (—^—-)) 

and by (1) from the definition of the WCGA and Lemma 6.10 from [IT] , p. 
343, we get 

\F fm _ l (<p m )\>tsvv\F fm _ l (g)\ = 

g€T> 



t sup 1^(0)1. 



From here we obtain 



\\fm\\ < ||/ m -i|| + inf(-A^ m + 2||/ m _ 1 ||p(A/||/ m _ 1 ||)). 
We discuss here the case p{u) < ■yu q . Using (13.51) we get 



Let Ai be a solution of 

At „ A g ( t\\u-x\\ q \^ 

2 1^1 fT77> A i 



Our assumption (12. 3 p gives 

II/4JI <^(||/m-i|| + e). 

Specify 



A : 

Then A < Ai and we obtain 



Vl67Ci|A| r C/9 
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II/-II < 11/^11 f 1 X ) + T' ll+t) . (3.6) 

Denote C\ := -4~ . This implies for m2 > mi > k 

2(167) « =T (C7iC0«' 



ll/m.|| < 11/^11(1-^/1^)^+ U ,; |2r l; (|]/ B ||+6). (3.7) 
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2ci(m 2 - mi) 



Define m := k and, inductively, 



m 



m j „ 1 + f3\A j \ rq ', j = l,...,n. 



At iterations from m^-i + 1 to rrij we use A = Aj and obtain from f)3.6p 

||/mj< 11/^11^ + 2(11^.11+6). 

We continue it up to j = L. Denote r\ := e _Cl/3 . Then 

L 

\\fm L \\< ||/ ft h X + 2^(||/ B ,||+e)^. 
i=i 

We bound the ||/fe||. It follows from the definition of /& that \\fk\\ is the error 
of best approximation of f by the subspace <3>fc. Representing f = f + fo~ f 
we see that ||/fc|| is not greater than the error of best approximation of / by 
the subspace $*; plus ||/o — /||- This implies \\fk\\ < \\fB \\ +£• Therefore we 
continue 

L 

<(ll/B || + 6) ?7 i + 2^(||/ B ,_ 1 ||(r ? 6)^6- 1 + 6 ?7 ^) 



<b- 1 \\fB L _ 1 \\((vb) L + 2j2(vb) L ~ j 

3=1 



2e 



1 — r] 



We will specify /3 later. However, we note that it will be chosen in such a 
way that guarantees r) < 1/2. Choose b = Then 

||/»J|< 11/^1186-^ + 46. (3.8) 

By (E3D we get 

ll/r^H < «7(||/mJ| +e) < [7(11^1186-^ + 56). 

We note that in the proof of Theorem 12.51 we use the above inequality with 
U = VK r < VK. If (I < lOUe then by (ETS1) 

11/mJI <CUe. 
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If II/bl-JI > lOUe then making (3 sufficiently large to satisfy 16Ue Cl/3 < 1 
so that = C3ln c ( i t/+1) , we get 

U(\\f BL - 1 \\^r ei/, + 5e)<\\fB Il . l \\ 

and therefore 

ll/r^H < ||/b l _J. 

This implies 

i-pm^i ^ |r^| 2^ 2 

We begin with / an d apply the above argument (with k — 0). As a result 
we either get the required inequality or we reduce the cardinality of support 
of / from \T\ = A to \Y mL ^\ < \T\ - 2 Ll ~ 2 , m Ll < (32 aL \ a := rq'. We 
continue the process and build a sequence ttil such that < (32 aLj and 
after iterations we reduce the support by at least 2 Lj ~ 2 . We also note 
that rriLj < (32 a K a . We continue this process till the following inequality is 
satisfied for the first time 

m Ll + ■ ■ ■ + m Ln > 2 2a /3K a . (3.9) 

Then, clearly, 

m Ll + ■ ■ ■ + m Ln < 2 2a+l (5K a . 

Using the inequality 

(a x + • • • + a n ) e <«? + ••• + <, a s > 0, 9 e (0, 1] 
we derive from (13. 9p 

2^1—2 | | ^> ^2^a(L\— 2) _|_ . . . _|_ 2 a (-k™~2)^ » 

> 2~ 2 (2 ail H V 2 aLn ) ° 

>2- 2 ((/3)- 1 (m Ll + --- + m L J)» > A. 

Thus, after not more than A := 2 2a+1 / 9A a iterations we recover / exactly 
and then ||/^|| <||/ - /|| <e. 

□ 
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4 Examples 



In this section we discuss applications of Theorems from Section 2 for specific 
dictionaries T>. Mostly, T> will be a basis ^ for X. Because of that we use 
m instead of K in the notation of sparse approximation. In some of our 
examples we take X = L p , 2 < p < oo. Then it is known that p(u) < 7« 2 
with 7 = (p — l)/2. In some other examples we take X = L p , 1 < p < 2. 
Then it is known that p(u) < , yu p , with 7 = 1/p. 

Example 1. Let \1/ be a uniformly bounded orthogonal system normal- 
ized in L p (Q), 2<p<oo,£7isa bounded domain. Then we have 

Ci(n,p)lhy 2 < HV'illp < c 2 (^p)||^|| 2 , j = 1, 2 ... . 

Next, for/ = ^Q(/)^ 

< \\fh\\ ^(si g n Q (/))^||^ll 2 2 ll2 < c 3 (n, P )\A\v 2 \\f\\ p . 

Therefore \1/ satisfies A3 with D = 00, V = C^(Q,p), r = 1/2. Theorem 12.71 
gives 

\\fo(t, P ,D)mln(m+l) ||p <Ca m {f ,*) p . (4.1) 

Example lq. Let ^ be a uniformly bounded orthogonal system normal- 
ized in L p (Q), 1 < p < 2, Q is a bounded domain. Then we have 

Ci(Q,p)||V;i|2 < \\4>j\\ P < C 2 (n,p)||^-|| 2 , J = 1,2. . . . 
Next, for / = £\ci(/)^ 

ieA ieA 
< \\f\\ p \\ ^(sign Q (/))^||^ll 2 2 || P ' < C^pM^WfW,. 

i<=A 

Therefore \I/ satisfies A3 with D = 00, V = C*4(fi,p), r = 1 — 1/p'. Theorem 
I2TT1 gives 

H/c(t,p,D)mP'-lln(m+l)llp < C(T m (f , ^) p . (4.2) 



17 



Example 2. Let \1/ be the normalized in L p , 2 < p < oo, real <i-variate 
trigonometric system. Then Example 1 applies and gives for any f G L p 

\\fc(t,p,d)mln(m+l)\\p < Cf7 m (/ , ^)p- (4.3) 

We note that (14. 3 p provides some progress in Open Problem 7.1 (p. 91) from 

Example 2q. Let \l/ be the normalized in L p , 1 < p < 2, real (i-variate 
trigonometric system. Then Example lq applies and gives for any fo G L p 

Wfc(t,p,d)mP'-^ln(m+l)\\p - C°"m(/o, *)p- (4.4) 

We need the concept of cotype of a Banach space X. We say that X has 
cotype s if for any finite number of elements G X we have the inequality 

^Average ± || ^±M,|| s j > C s ^E ||w;|| s j . 

It is known that the L p spaces with 2 < p < oo have cotype s = p and L p 
spaces with 1 < p < 2 have cotype 2. 

Remark 4.1. Suppose T> is (K , K) -unconditional with a constant U. As- 
sume that X is of cotype s with a constant C s . Then T> has the Nikol'skii-type 
i\X property with parameters K, 1 — l/s and C\ = 2UC~ l . 

Proof. Our assumption about (K, i^)-unconditionality implies: for any A, 
\A\ < K, we have 

|| < 2U\\ 22i Xi 9i\\- 

ieA ieA 

Therefore, by s-cotype assumption 



.1, ; 



ieA ieA 

This implies 

n TT^,-\ I A la II 

, %i9i I 



E ^ (e w s ) 7 ^ wc-w-V'w e 



□ 
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Example 3. Let X be a Banach space with p(u) < 7W 9 , 1 < q < 2 and 
with cotype s. Let \I/ be a normalized in X unconditional basis for X. Then 
U < C(X, and * satisfies A2 with D = 00 and any if. 

By Remark 14.11 \1/ satisfies Al with r = 1 — -. Theorem 12.41 gives 

||<Ca m (/ 0j *). (4.5) 

Example 4. Let ^ be the normalized in L p , 2 < p < 00, multivariate 
Haar basis % p = % p x ■ • • x Hp. It is an unconditional basis. Also it is known 
t licit Lp space with 2 < p < 00 has cotype s = I /p. Therefore, Example 3 
applies in this case. We give a direct argument here. It is an unconditional 
basis and therefore U < C(p,d). Next, for any A 

\\J2 x i H iAp> c (p> d ) few p ) >c7(p,d)|A|H^| Zi |. 

i£A \i£A / i&A 

Therefore, we can take r = ~. Theorem 12.41 gives 

\\fc(t, P ,d)m?/p'\\p ^ ( ^ cr m{fo,'Hp)p- (4.6) 

Inequality (14. 6 j) provides some progress in Open Problem 7.2 (p. 91) from 
[TO] in the case 2 < p < 00. 

Example 4q. Let ^ be the normalized in L p , 1 < p < 2, univariate 
Haar basis %p = {if/ >p }/, where H^ p the Haar functions indexed by dyadic 
intervals of support of Hj p (we index function 1 by [0, 1] and the first Haar 
function by (0, 1]). Then for any finite set A of dyadic intervals we have for 
f = EiCi(f)Hj, p 



Therefore, 



■ I — 2 

p\\2 ■ 

ieA leA 



J2\ci\<\\f\\ P \\rA\\ P >- 

leA 

It is easy to check that 

||ffi I pUI#i lP ||a 2 = |J|-Vp| 7 |W|j|-(i-2/p) = L 
By Lemma 1.23, p. 28, from [TT] we get 

WfAh<C{p)\A\^'. 
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Thus 

J2\ c i\<C(p)\A\^'\\f\\ p . 
ieA 

This means that T-L p satisfies A3 with V = C(p) and r = 1/p'. Also it 
is an unconditional basis and therefore satisfies A2 with U = C(p). It is 
known that L p space with 1 < p < 2 has modulus of smoothness p{u) < r yu p . 
Therefore, Theorem 12.81 applies in this case and gives 

||/c(t,p)m||p — C^m(fo,'Hp)p- (4.7) 

Inequality (" 14. T[) solves the Open Problem 7.2 (p. 91) from [10] in the case 
1 < p < 2. 

Example 5. Let X be a Banach space with p{u) < ju 2 . Assume that \1/ 
is a normalized Schauder basis for X. Then for any / = Yli^if)^ 

5>(/)i<c(*)wii. 

i£A 

This implies that \& satisfies A3 with D = oo, V = C(^>), r = 1 and any T. 
Theorem 12.71 gives 

\\fo(t ,X^)m? lnm || <C<7 m (/„,*). (4.8) 

We note that the above simple argument still works if we replace the as- 
sumption that \I/ is a Schauder basis by the assumption that a dictionary T> 
is (1, /^-unconditional with constant U. Then we obtain 

\\fc{t,y,u)K*in K \\ <Ca K (f ,V), for K + C(t, 7 , U)K 2 In K < D. 

Example 5q. Let X be a Banach space with p(u) < 1 < q < 2. 
Assume that \1/ is a normalized Schauder basis for X. Then for any / = 

1>(/)|<C7(*)|4||/||. 

ieA 

This implies that \& satisfies A3 with D = 00, V = Cfif), r = 1 and any T. 
Theorem 12.71 gives 

\\fc(t ,X,-V)mi' lnm 

|| <C<7 m (/ ,*). (4.9) 
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We note that the above simple argument still works if we replace the as- 
sumption that \I/ is a Schauder basis by the assumption that a dictionary T> 
is (1, /^-unconditional with constant U. Then we obtain 

ll/c(* >7l ^'iaxll<C^(/o,P), for K + C&wV)K<\nK<D. 

We now discuss application of Theorem 12.11 to quasi-greedy bases. We 
begin with a brief introduction to the theory of quasi-greedy bases. Let X be 
an infinite-dimensional separable Banach space with a norm || • || := || • ||x an d 
let $ := {ipm}m=i b e a normalized basis for X. The concept of quasi-greedy 
basis was introduced in [3]. 

Definition 4.1. The basis \1/ is called quasi- greedy if there exists some con- 
stant C such that 

sup||G m (/,tt)|| <C\ 



Subsequently, Wojtaszczyk [13] proved that these are precisely the bases 
for which the TGA merely converges, i.e., 

lim G n (f) = f. 

n— >oo 

The following lemma is from [TJ (see also [3]). 

Lemma 4.1. Let ^ be a quasi-greedy basis of X . Then for any finite set of 
indices A we have for all f G X 

ws A (f,n\<cH\A\ + i)\\f\\. 

We now formulate a result about quasi-greedy bases in L p spaces. The 
following theorem is from [12] . We note that in the case p = 2 Theorem 14.11 
was proved in [T3]. Some notations first. For a given element / 6 X we 
consider the expansion 

oo 
k=l 

and the decreasing rearrangement of its coefficients 

Mf)\ > \c k2 (f)\ > - • 

Denote 

a n (f) ■= |c fcn (/)|. 
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Theorem 4.1. Let ^ = {?/>m}m=i be a Quasi- greedy basis of the L p space, 
1 < p < oo . Then for each f G X we have 

oo 

C 1 {p)sn V n 1 ^a n {f)<\\f\\ P <C 2 {p)Y,n- l/2 a n U), 2 < p < oo; 

n=l 

oo 

C^supn 1 / 2 ^/) < < C^Ys^^nif), K P <2. 

n 

n=l 

Example 6. Let \1/ be a normalized quasi- greedy basis for L p , 2 < p < oo. 
Theorem 14.11 implies for any / = Xli^l/)^ 

\A\ \A\ 

£ Hf)\ < J2aM < cM-'Y^wfWp < ^)i^r i/p imi P . 

i£A n=l n=l 

This means that \P satisfies A3 with D = oo, V = C(p), r = 1 — K Theorem 
12.71 gives 

ll/<7(t#)m2(i-Vi0lii(m+l)ll - Ccr m (f ,^). (4-10) 

Example 6q. Let ^ be a normalized quasi-greedy basis for L p , 1 < p < 
2. Theorem 14.11 implies for any / = 5^ Q (/)?/>« 

\A\ \A\ 

J2h(f)\ < E a «(/) ^ ^(p)~ iy E n ~ 1/2 \\f\\p ^ c (p)\ A \ 1/2 \\f\\p- 

i£A n=l n=l 

This means that \I> satisfies A3 with D = oo, V = C(p), r = 1/2. Theorem 
12.71 gives 

,p)mP r / 2 ln(m+l) 

||<C(7 w (/ ,*). (4.11) 

Example 7. Let \I/ be a normalized uniformly bounded orthogonal quasi- 
greedy basis for L p , 2 < p < oo. For existence of such bases see [6]. Then 
orthogonality implies that we can take r = 1/2. We obtain from Lemma I4TT1 
that \I/ is (K, oo) unconditional with U < C\n(K + 1). Theorem 12.81 gives 

\\fc(t ,p,*)mlnln(m+3) \\p <Ca m {f ,*) p . (4.12) 

Example 7q. Let $ be a normalized uniformly bounded orthogonal 
quasi- greedy basis for L p , 1 < p < 2. For existence of such bases see [6]. 
Then orthogonality implies that we can tame r = 1/2. We obtain from 
Lemma \A. II that \I> is (K, oo) unconditional with U < Cln(iT + 1). Theorem 
12.81 gives 

\\fc(t,p,y)mP'/ 2 lnln(m+3)\\p - C (T m(/o, ^)p- (4-13) 
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5 Discussion 



We study sparse approximation. In a general setting we study an algorithm 
(approximation method) A = {A m (-, D)}~ =1 with respect to a given dic- 
tionary V. The sequence of mappings A m (-,V) defined on X satisfies the 
condition: for any / G X , A m (f,T>) G £ m (X>). In other words, A m provides 
an m-term approximant with respect to T>. It is clear that for any / G X 
and any m we have 

\\f-A m (f,V)\\>a m (f,V). 

We are interested in such pairs (V, A) for which the algorithm A provides 
approximation close to best m-term approximation. We introduce the corre- 
sponding definitions. 

Definition 5.1. We say that V is a greedy dictionary with respect to A if 
there exists a constant Co such that for any f G X we have 

\\f-A m (f,V)\\<C a m (f,V). (5.1) 

If V is a greedy dictionary with respect to A then A provides ideal (up 
to a constant C ) m-term approximations for every / G X. 

Definition 5.2. We say that V is an almost greedy dictionary with respect 
to A if there exist two constant C\ and C2 such that for any f G X we have 

\\f-A Cim (f,V)\\<C 2 a m (f,V). (5.2) 

If V is an almost greedy dictionary with respect to A then A provides 
almost ideal sparse approximation. It provides Cim-term approximant as 
good (up to a constant C2) as ideal m-term approximant for every / G X. 
We also need a more general definition. Let 4>{u) be a function such that 
4>(u) > 1. 

Definition 5.3. We say that V is a (p-greedy dictionary with respect to A if 
there exists a constant C3 such that for any f G X we have 

||/-^Mm(/,Z>)|| <C 3 a m (f,V). (5.3) 
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If T> = \1/ is a basis then in the above definitions we replace dictionary 
by basis. In the case A = {G m (-, ^)}^L=i is the TGA the theory of greedy 
and almost greedy bases is well developed (see [H]). We present two results 
on characterization of these bases. A basis \I> in a Banach space X is called 
democratic if there is a constant C(^) such that 



if \A\ = \B\. This concept was introduced in |3]. In [2] we defined a demo- 
cratic basis as the one satisfying (15. 4p if \A\ < \B\. It is known that for 
quasi-greedy bases the above two definitions are equivalent. It was proved 
in jl] (see Theorem 1.15, p. 18, [TT]) that a basis is greedy with respect to 
TGA if and only if it is unconditional and democratic. It was proved in [2] 
(see Theorem 1.37, p. 38, [11]) that a basis is almost greedy with respect to 
TGA if and only if it is quasi-greedy and democratic. 

Example 4q is the first result about almost greedy bases with respect 
to WCGA in Banach spaces. It shows that the univariate Haar basis is 
an almost greedy basis with respect to the WCGA in the L p spaces for 
1 < p < 2. Example 1 shows that uniformly bounded orthogonal bases are 
0-greedy bases with respect to WCGA with <j>(u) = C(t,p, D) ln(w + l) in the 
L p spaces for 2 < p < oo. We do not know if these bases are almost greedy 
with respect to WCGA. They are good candidates for that. 

It is known (see [TT], p. 17) that the univariate Haar basis is a greedy 
basis with respect to TGA for all L p , 1 < p < oo. Example 4 only shows 
that it is a 0-greedy basis with respect to WCGA with (j)(u) = C(t,p)u 1 ~ 2 / p 
in the L p spaces for 2 < p < oo. It is much weaker than the corresponding 
results for the 7-L p , 1 < p < 2, and for the trigonometric system, 2 < p < oo 
(see Example 2). We do not know if this result on the Haar basis can be 
substantially improved. At the level of our today's technique we can observe 
that the Haar basis is ideal (greedy basis) for the TGA in L p , 1 < p < oo, 
almost ideal (almost greedy basis) for the WCGA in L p , 1 < p < 2, and that 
the trigonometric system is very good for the WCGA in L p , 2 < p < oo. 

Example 2q shows that our results for the trigonometric system in L p , 
1 < p < 2, are not as strong as for 2 < p < oo. We do not know if it is a lack 
of appropriate technique or it reflects the nature of the WCGA with respect 
to the trigonometric system. 

We note that properties of a given basis with respect to TGA and WCGA 
could be very different. For instance, the class of quasi-greedy bases (with 




(5.4) 
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respect to TGA) is a rather narrow subset of all bases. It is close in a certain 
sense to the set of unconditional bases. The situation is absolutely different 
for the WCGA. If X is uniformly smooth then WCGA converges for each 
/ G X with respect to any dictionary in X. Moreover, Example 5q shows 
that if X is a Banach space with p(u) < 7M 9 then any basis \1/ is 0-greedy 
with respect to WCGA with <j>(u) = C(t, X, ^>)u q '~ l ln(u + 1). 

It is interesting to compare Theorem 12.31 with the following known re- 
sult. The following theorem provides rate of convergence (see [TTJ, p. 347). 
As above we denote by A\(T>) the closure in X of the convex hull of the 
symmetrized dictionary T> ± := {±g : g G T>}. 

Theorem 5.1. Let X be a uniformly smooth Banach space with modulus of 
smoothness p{u) < r yu q , 1 < q < 2. Take a number e > and two elements 
fo, f e from X such that 



Both Theorem 15. II and Theorem 12.31 provide stability of the WCGA with 
respect to noise. In order to apply them for noisy data we interpret fo as 
a noisy version of a signal and f e as a noiseless version of a signal. Then, 
assumption f e /A(e) G A\{T>) describes our smoothness assumption on the 
noiseless signal and assumption f e G describes our structural assump- 

tion on the noiseless signal. In fact, Theorem 15.11 simultaneously takes care 
of two issues: noisy data and approximation in an interpolation space. The- 
orem 15.11 can be applied for approximation of fo under assumption that fo 
belongs to one of interpolation spaces between X and the space generated 
by the y4i(D)-norm (atomic norm). 

Concluding, we briefly describe the contribution of this paper. First, we 
present a study of the Lebesque-type inequalities with respect to the WCGA 
in Banach spaces with p(u) < 1 < q < 2, under conditions Al and 
A2. In the case q — 2 it has been done in [5]. The case 1 < q < 2 uses 
the same ideas as in j5]. Second, we introduce a new condition A3 and 
study the WCGA with respect to dictionaries satisfying either A3 or A2 
and A3. Condition A3 and a combination of A2 and A3 turn out to be 



r/A(e) G A 1 (V), 



with some number A(e) > 0. Then, for the WCGA we have 
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more powerful in applications than Al combined with A2. Third, we apply 
the general theory developed in Sections 2 and 3 for bases. Surprisingly this 
technique works very well for very different bases. It provides first results 
on the Lebesque-type inequalities for the WCGA with respect to bases in 
Banach spaces. Some of these results (for the Hp, 1 < p < 2, and for the 
TWpi 2 < p < oo) are strong. This demonstrates that the technique used is 
an appropriate and powerful method. 
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